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ABSTRACT 

This literature review and evaluation "tool kit" 
provides a summary of the research literature about the evaluation of 
large-scale intervention policies like the Blueprint for the Future 
program of the Scarborough (Ontario, Canada) Board of Education. This 
report is intended to assist in creating a realistic plan for the 
Blueprint evaluation. An introduction covers the general concerns 
that must be considered when evaluating program implementation. 
Evaluating process versus outcome, fidelity versuc impact, and 
program components and outcomes is considered. General evaluative 
methods are reviewed. The rest of the report deals more specifically 
with the evaluation of the Blueprint program process and outcomes. 
Sections on process and outcomes each di.';cuss: (I) consultation; (2) 
intervention; and (3) staff development. Appendixes list three 
instruments that are available for summative and formative evaluation 
of Blueprint in the area of teacher interventions, role conflict and 
ambiguity, and attitudes toward mainstreaming. A fourth appendix 
provides examples of items from response choice questionnaires. 
(SLD) 
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PREFACE 



The mission statement for the "Blueprint for the Future" states the following: 

Special Education Services of the Scarborough Board of 
Education is committed to sharing expertise and to 
providing direct support to students, school personnel 
and families in ^order to ensure that every student with 
special needs is provided with the most enabling learning 
environment. 

Underlying this clear and straightforward statement is a complex set of strategic 
policies, objectives, strategies and action plans guiding the implementation of the plan 
over a projected five- to seven-year period. 

One of the policy strategies put forward by the plan, states that "Systematic and 
ongoing evaluation shall be an essential element for all Special Education Services". This 
report is one result of this pohcy. It is a hterature review focusing on the evaluation 
component of implementing a large-scale integration program such as Blueprint\ It has 
been compiled to offer the Blueprint steering committee a comprehensive overview of 
those factors which have been identified in other research as influential in implementing 
team-centered consultation models, as well as some suggestions about data-collection 
methods and instruments. 

The introduction to the report covers the general concerns that have to be 
considered when evaluating program implementation. The rest of the report is divided 
into two sections; one deahng with program process and the other with program 
outcomes. Research directed at process attempts to identify what program activities are 
actually taking place, as well as what barriers or facilitators are affecting the 
implementation process. Research directed at outcomes attempts to assess the effect of 
the program on all participants - practitioners as well as chents. 

These two sections (Process Evaluation and Outcome Evaluation) are further 
divided into sections representing the three main program components of Blueprint: 
Consultation, Intervention, and Staff Development. Suggestions for process or outcome 
evaluation are organized within these program component headings. Some available 
survey instruments smtable for use with Blueprint are included in the Appendices. 



^The term "integration" will be used in this report to refer to the type of program 
that the Blueprint is trying to implement. The term "mainstreaming" is often used in the 
research literature, and is retained here in the original titles of questionnaires or articles. 



INTRODUCTION 

I 

Evaluation of ^ large-scale policy change like Blueprint can take many forms and 
requires many decisions. This introduction provides brief descriptions of some of the 
evaluation concerns ttat need to be considered, and decisions that need to be made, in 
planning an evaluation of Blueprint. 

Three major questions have to be answered when planning any evaluation: 

4 What to evaluate? A large-scale and comphcated policy initiative such as 
the Blueprint contains numerous factors that may influence the success of 
the program. A framework is required to organize these factors. 

♦ How to evaluate? Choosing appropriate instruments enables quality data 
to be collected. Great care has to be taken to make data collection practical, 
non-intrusive and non-threatening. 

♦ What to do with the results? How results are fed back to program 
decision-makers, as well as practitioners in the field, determines hov/ much 
this information will actually be used to make adjustments. 



WHAT TO EVALUATE 



PROCESS VERSUS OUTCOME 

There are two types of evaluation data that can be collected: process and outcome. 
Process evaluation has to do with describing in detail what is actually happening as a 
program is being implemented. Process evaluation is done to verify that a program is 
indeed operating as it was intended and to determine the factors that influence 
implementation. Outcome evaluation establishes whether a program is having any effect 
on the participants (i.e., practitioner, student or parent). 

An evaluation of Blueprint should include both process and outcome data collection. 
The value of outcome data is obvious - the bottom line is to find out whether or not the 
program is having, an effect. The value of process data usually becomes apparent when 
an. explanation is needed to account for outcome differences; for example, to explain why 
the program worked at one school but not at another, or to provide descriptions to others 
who want to duplicate the program. 



FIDELITY VERSUS IMPACT 

When a program's goals, procediures and expected outcomes are stipulated in detail, 
it is possible to assess implementation in terms of the extent to which a program in 
practice matches the program as intended. The more fidelity to an original conception 
is sought after, the less local initiative is encouraged. Blueprint, by contrast, exemplifies 
an organizational development approach to program implementation which encourages 
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local initiative and is designed to increase individual and organizational capacity to 
function with new demands and a new way of doing things; for example, mth a new 
model of service deUvery. While all participants take part in the same preparatory 
inservice training, the actual form that collaborative consultation v/ill take depends on the 
decisions of each school support team as it responds to the local school conditions and 
problems. The focus in evaluating such an approach, therefore, should be less on fideUty 
of implementation and more on the relative impact the program is having in each school 
on meeting the needs of students with special needs. 

Another reason for emphasizing program effectiveness and not fidelity is the fact 
that while Blueprint advocates a consultation approach to integration, no "pm'e" form of 
service delivery is being implemented. Maria Yau of the Toronto Board of Education 
identified four "pure" models of service deUvery for meeting the needs of learning-disabled 
students other than pull-out programs (Yau, 1988). These were: the consulting teacher 
model, the consulting and resource teacher model, the problem-solving team model and 
the team-teaching model. While research differentiates between these in an attempt to 
isolate what "pure" model works best, these models are usually combined in practice in 
a more eclectic approach to service delivery. 

With a program hke Blueprint which emphasizes local initiative, as well as an 
eclectic, problem-solving approach to service deUvery, it makes more sense to evaluate 
the program in terms of impact rather than fideUty. 

PROGRAM COMPONENTS 

Evaluation of Blueprint will require "simultaneous and systematic attention to 
multiple influences and multiple outcomes" (Littie, 1982). Things are happening at a 
number of levels [board of education, school, classroom, individual (e.g., teacher, support 
staff, student)], and there are different influences and outcomes at each level. 

In an ideal world, with unlimited time and resources, it would be possible to carry 
out a comprehensive and complete evaluation of Blueprint. However, under the 
constraints imposed by the real world, it is necessary to set priorities and to develop a 
strategy to identify the important core components to be evaluated. 

The Blueprint plan has three main components: 

t Consultation 

^ Intervention 

♦ Staff Development 

The core component of the program is the consultation process involving the school 
support teams, the resource/consulting teachers and the regular teachers. Undergir^g 
the consultation process is the preparatory inservice training dehvered by cenirvl office. 
As well, the school support teams have the responsibility of identifying the ongoing needs 
for inservice at the school level and to develop the necessary networking to meet these 
needs. Finally, the output of the consultation is classroom intervention - putting an 
intervention plan by the regular classroom teacher into practice. 
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Analyzing a program into its components is important for interpreting results. It 
becomes clearer what to do with particular data when it is clear which aspect (component) 
of a complex process the data is reflecting. In the case of Blueprint, process and outcome 
data will reflect the status of the consultation, intervention or staff development process. 



POSSIBLE OUTCOMES 

One of the most important decisions to be made in carrying out a program 
evaluation involves the choice of outcomes by which to judge the success of a program. 
In general, there are two types of outcomes: 1) changes that occur in the participants 
themselves, both practitioner and student, and 2) changes that occm- ia organizational 
capacity and functioning. 

This section provides a description of the most important outcomes identified in 
the evaluation literature on the integration of special education students into regular 
classrooms. 

A- Outcomes as Changes in the Participants 

(i) Practitioner Outcomes: Attitudes Towards Integration 

Both training and experience have been showvi to influence attitudes towards 
integration. Attitudes are understood to be effects rather than causes in this research. 
In other words, attitudes are seen to be a sort of baxoraeter which reflect vaiious degrees 
of good or bad experience with integration, or relative levels of expertise and comfort 
with special needs. 

Inservice training can influence teachers' attitudes towards integration, but only 
if this training is fairly intensive. Larrivee (1981) found that after an intensive six- 
week (4 hours a day) summer workshop, participants had more positive attitudes than 
either a random sample of teachers who received no training, or a group of teachers who 
received once-a-month inservice (2 hours each time) for eight months. The latter two 
groups did not differ in their attitudes towards integration. Interestingly, the item on the 
questionnaire which revealed the greatest difference between the gi^oups stated that 
"regular teachers possess a great deal of the expertise necessary to work with special 
needs students". 

Leyser (1988) found similar results with student teachers undergoing a preservice 
program involving coursework and field experience over two years, compared to a control 
group of student teachers. She found that these two groups differed not only m attitudes 
but in classroom effecti.veness with exceptional children. The students with speaal needs 
evidenced less on-task behaviour in the classrooms taught by the control teachers. 

Larrivee (1981) identified three features necessary for training to actually have an 
influence on attitudes: 

♦ Actual contact with handicapped or other children with special 
needs during the training. 



♦ 



A focus on knowledge and skill attainment. 



♦ The presence of strong supportive assistance for teachers. 

Attitudes can reflect differences in actual expenence with teaching exceptional 
children. For exajaple, school administrators are usually the most positive about 
integration, classroom teachers are the least positive, and special education teachers fall 
somewhere in between (Garvar-Pinhas and Schmelkin, 1989). Regular teachers who 
report more experience with disabled students also tend to report more positive attitudes 
towards integration (Hayes and Gimn, 1988). 

Attitudes towards integration also vary with the level of learning disabiUty 
or handicap considered. Berryman (1988, 1989) found that both teachers and the lay 
pubHc felt positive towards integrating students with mild disabilities such as stuttering, 
or vath handicaps that do not int>erfere with their learning abiHty (e.g., diabetes, 
epilepsy). Negative attitudes were felt for integrating students with serious behavioin: 
disorders or discipline problems. Neutral attitudes ("luicertain") were felt toward 
integrating blind, deaf, physically-handicapped and educable mentally retarded students. 
Center and Ward (1987) found similar results in Australia. They argue that the positive 
attitude towards integrating students with mild disabilities is based on the assumption 
that these students will not require extra instructional or management skill on the part 
of the classroom teacher. 

Center and Ward (1987) also found that regular and resource teachers disagreed 
regarding the integration of students who required additional educational or Lehavioural 
strategies in teaching (e.g., students with impaired language skills, intellectual disability, 
short attention span and aggressive tendencies). They make an interesting observation 
about the effect of labelling students on regular teachers' attitudes towards integration: 

"Although children with short attention span, hyperactivity or individualized 
instructional needs always have been found within the regular classroom, 
when identified as disabled, they appear to generate some anxiety 
concerning the suitability for mainstreaming." 

Regular classroom teachers, therefore, tend to be reactive to integration because 
of the anxieties associated with it. Even if they agree with the philosophy behind 
integration, teachers are very aware that it will "make their jobs more difficult and 
frustrating" (Hayes and Gimn, 1988). Center and Ward (1987) found that, if regular 
teachers feel that they lack the expertise or the support necessary to deal with special 
needs students, they will tend to overreact by being conservative in their teaching 
approach with such students, rather than extending their skills. The resource teacher can 
have an important moderating influence on the regular teacher by providing the necessaiy 
support in the classroom. 

(ii) Practitioner Outcomes: Skill Dovelonment 

Skill development is an obvious outcome expected of staff development programs. 
Yet, there are few examples in the research Hterature in which skill developraent is 
directly measured as an outcome. This is due to the difficulty in testing skill directly. 
It would require either a time-consuming simulation or ongoing observation and 
evaluation of staff by an experienced practitioner. Typically, skill development is 
evaluated indirectly; for example, by combining process measures (practitioners' practices) 
with outcome measures (client outcomes). If the process measures confirm that certain 
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practices indeed occurred, and the outcome measures indicate client improvement, then 
v/e can conclude that adequate skill was used in carrying out the interventions or 
treatments. 

Another method of indirectly evaluating skill development is to elicit a self-report 
or self-assessment of performance in different skill ax^eao. In this case, the same checklist 
that is used in establishing frequency of use of particular practices or activities is used 
to ask the same respondents to indicate the practices in which they feel competent and 
the ones where 'they require farther training. In evaluating Blueprint, information 
gathered to describe the process caoi also be used to identify areas of accomplished skill 
development or areas where further training is required to enhance skills in the following 
areas: school support team procedures, consultation practices and classroom intervention 
practices. In effect, the formative evaluation of practitioners' skill development resembles 
an ongoing needs assessment. 

(iii) Practitioner Outcomes: Perceptions About the Consultation and 
Intervention Process 

During the first few years of Blueprint it is probably important to find out what 
the regular teachers, resource teachers and school support team members think of how- 
well the consultation process is working. A vital feature of the consultation process is 
the relationship between the consulting/resource teachers and the regular teachers. This 
relationship involves a number of issues which, if not dealt with, can lead to conflict 
rather than collaborative consultation. Glatthom (1990) lists the following possible 
sources of conflict between consultants and consultees: 

♦ Different perceptions of the role of the special educator. 

♦ Different frames of reference concerning teaching and learning; 
for example, individuafized versus group instruction; or an 
emphasis on a wide range of learning and coping skills versus 
academic skills and content. 

♦ Different work-related languages (terminology, jargon). 

♦ Different methods and materials. 

f Different perceptions of each other's competence in working 
with mildly-handicapped students. 

The research literature identifies a number of different ways of looking at the 
consultation process: role perceptions held by the participants, expert versus collaborative 
consultation, satisfaction and stated concerns and intervention acceptability. 

Role Perceptions: Crane and Iwanicki (1986) investigated the relationship 
between the role conflict and ambiguity felt by special educators, and job stress and burn- 
out They found that role conflict was closely linked with emotional exhaustion and 
depersonalization, while role ambiguity was linked to a relative feeling of personal 
accomplishment. They also found that special education teachers m self-contained 
classrooms showed higher levels of bum-out than did resource room teachers. Measures 
of the amount of role conflict and ambiguity a teacher experiences is Hnked, therefore, to 
an important outcome - the relative feeling of job satisfaction which can not only prevent 
burn-out, but also be a motivator in doing a better job. At the very least, if teachers 
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evidence high role conflict and ambiguity in a program, it is fairly likely that the 
atmosphere is not ripe for acquiring new skills. 

Expert versus CoDaborative Consioltation: Another way of looldng at role 
perceptions, which does involve the regular teachers, is the perceptions that regular 
teachers and consultants have of each other's roles. A distinction is made between an 
expert model and a collaborative model of consultation. PhilUps and McCullough (1990) 
elaborate as follows: 

In expert models, the relationship between the consultant and consultee is 
hierarchial: the consultant serves as the expert and tbe consuitee as the 
recipient of the expertise.. '.In the collegial relationship, peers who share 
some basic body of knowledge join in exchanging specific ideas and 
experiences to solve problems encountered in areas of mutual understanding 
or interest. 

Teachers prefer a collaborative model of consultation (Pryzwansky and White, 
1983). Because the consultation process is basically one of help-seeking by the consultee 
and help-giving by the consultant, this process is not an emotionally-neutral one, but one 
infused with anxiety, resistance and power struggles. Witt and Martens (1988) identify 
two major problems with the use of a purely expert model of consultation in education: 
1) Teachers begin to expect that the "experts" will have all the answers; and, 2) The help 
dehvered, while effective in the short term for the student, may have long-term negative 
effects for the teachers; namely, reducing their control and responsibility over teaching, 
reducing their motivation to acquire new skills, and lowering their perceived self-efficacy. 
So it is easy to see why teachers' caution, and even resistance, is to be expected. As 
Margohs and McGettigan (1988) point out, teacher resistance can be interpreted as the 
kind of caution to be expected as the legitimate reaction of a professional who is 
concerned with maintaining autonomy in how they teach their students. 

Satisfaction and Stated Concerns: Role perceptions deal with the most abstract 
level at which people perceive each others' responsibilities and competence. There is also 
the more practical level of how well the program appears to be working, hew pai-ticipants 
feel the implementation is going and how satisfied they are with the program. Typically, 
the most direct way to get this information is an open-ended q iestionnaire or interview 
in which respondents are simply asked to state in their own words how satisfied they are 
with a program, and what their concerns are. 

Chalfant and Van Dusen Pysh (1989) used such an open-ended questionnaire to 
collect information about teachers' reactions to consultation with Teacher Assistance 
Teams Even though TATs are composed only of regular teachers, the general response 
categories that Chalfant and Van Dusen Pysh constructed to organize the teachers 
comments probably represent issues and concerns about consultation in general. Tbey 
organized the positive comments into six categories of statement of satisfaction: 

♦ Effectiveness of group problem solving to generate useful 
strategies. 

♦ Provision of moral support and reinforcement of teachers by 
team members. 



Improvement of student performance and behaviour. 



♦ Facilitation of commuiucation among faculty and improvement 
in the use of interpersonal skills among teachers, 

♦ Improvement • of skill and comfort level in analyzing and 
imderstanding classroom problems. 

♦ Assistance in expediting the referral process (evaluation and 
placement). 

Negative comments were classified iato eight categories of statement of concern: 

♦ Insufficient time allocated for team meetings and 
implementation. 

♦ Failure to generate useful iatervention strategies. 

♦ Interference with special education referral process. 

♦ Lack of faculty readiness to initiate the team. 

♦ Little or no impact on student performance or behaviour. 

♦ Inefficient organization and management of team meetings. 

♦ Too much paperwork. 

♦ Confusion about the role of TAT and other building (i.e., within-the-school) 
programs. 

Concerns voiced in other surveys (Center and Ward, 1987; Bauwens et al., 1989) included: 

♦ Anxiety about own lack of skills involving children with special 
needs. 

♦ Not enough time for individual attention for these students; 
classes too large. 

♦ Lack of back-up support fi^om support services. 

♦ Difficulty of developing cooperative working relationships with 
others, especially if they have different training. 

♦ Anticipation of iacreased workload in having to teach students 
with special needs. 

Intervention Acceptability: Of all the regular teachers' concerns, one concern 
has received particular emphasis in the research Hterature on consultation. This has to 
do with the critical issue of teachers' judgments of the "acceptabihty" of interventions 
developed in consultation with special education or other support staff. 

Martens et al. (1986, 1989) define iatervention acceptability as "judgments of 
whether treatment is fair, reasonable or intrusive, appropriate for the problem, and 
consistent with notions of what treatment should be". In one study they mailed a 65- 
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item que::itionnaire ("Classroom Intervention Profile") to about 2500 regular and special 
education teachers in Iowa (Martens et al., 1986). The teachers were asked to rate each 
described intervention on a five-point Likert scale on three dimensions: relative 
effectiveness, ease of use and frequency of use. The results showed that teachers 
intuitively organize intervention strategies into separate categories. Two types of strategy 
were rated highly on all three dimensions: "Redirection of the student toward 
appropriate behaviour via a signal"; and "Manipulation of previously contracted material 
rewards". Martens et al. (1986) conclude that teachers prefer intervention strategies that 
reinforce appropriate behaviour rather than punish inappropriate behaviour. Also, 
teachers prefer signalling strategies which require Httle in the way of teacher time and 
resoiirces, rather than strategies, such as point reward systems, that require extra time. 
An interesting result was teachers' response to strategies involving consultation with 
"specialists". While strategies resulting from consultation were seen as effective, they 
were rated low in ease of use. Martens et al, (1986) conclude that teachers are 
particularly sensitive to demands on their time, including meetings with other school 
personnel. 

A different methodologj^ was used by Martens and Meller (1989) to study the effect 
of differences in child or classroom characteristics on judgments of intervention 
acceptability. They presented student teachers with two vignettes describing a student 
with problematic behaviour. One vignette contained an additional description of the child 
himself/herself (level of intelUgence, degree of popularity and long-standing versus short- 
term problem), while the second vignette described characteristics of the classroom 
(regular or self-contained, size and student composition). The teachers were then asked 
to fill out a 15-item Likert scale instrument called the Intervention Rating Profile which 
measures different aspects of intervention acceptability. The results showed that 
teachers' ratings were affected by child characteristics but not by classroom 
characteristics, revealing a "child-centered" as opposed to "environmental" perspective on 
the causes of students' problems. They also foxmd that teachers anticipate little success 
in treating long-standing problems, suggesting that "during consultation, the extent to 
which teachers perceive themselves as competent in cbnnging children's behavioxir may 
be an important issue" (Martens and Meller, 1989). Any evaluation of teachers' 
judgments of intervention acceptabihty should take into consideration mitigating 
circumstances such as the particiilar academic or behavioral problems encountered. 

While Martens et al. ;i986) have produced much important information about 
intervention acceptability, the most promising instrument for estabUshing both degree of 
use and judged "reasonableness" (i.e., acceptabihty) is the Teacher Intervention 
Questionnaire developed by Johnson and Pugach (1990). (This questionnaire is described 
later in the Process Evaluation section.) Johnson and Pugach stress that it is important 
to compare judged reasonableness to actual reported use. Any discrepancies indicate 
areas in which teachers may require more information, more inservice training or more 
encouragement. For example, they found that teachers rated "compiling data about 
behaviour problems" and "classroom demonstrations" as very reasonable, but infirequently 
done in actuality, 

Johnson and Pugach (1990) also interviewed the respondents to aftk them "what 
factors they considered when determining whether an intervention was reasonable or 
unreasonable". The responses fell under nine general categories: 
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Sends wrong message (i.e., intervention could commimicate 
to students that they got away with something). 



Too time consuming. 

Not fair to other children (i.e., preferential treatment). 
Makes problem worse. 
Doesn't work on the actual problem. 

Another strategy in 'teacher's experience has been more successful. 

Diminishes teacher authority (i.e., seeking help thought of as admitting 
defeat). 

Organizational rules/structures discourage use (i.e., the way a classroom or 
school is organized and managed prevents intervention from being fully 
implemented). 

Intervention inappropriate given child's age or severity of the problem. 

Johnson and Pugach (1990) concluded that, in general, teachers were more able to 
state why they had not selected an intervention than why they had selected it. 

A similar list of teachers' concerns about the "fea.sibiHty" of intervention was 
drawn up by Phillips and McCullough (1990) as part of a paper on the necessity of 
having a collaborative ethic as a basis on which to ground consultation-based 
programming They note that a persistent criticism of consultation by regular teachers 
is that it often generates "unreahstic solutions", and that consultants fail to recognize 
"the complexity of regular classrooms and the pressures imposed on regular teachers . 
They point out that there is a high probabiUty that teachers wiU "reject any intervention 
outright, before contemplating its merits, if they beUeve that the consultant has not 
adequately considered feasibility". 

(ivl Student Outcomes. 

Student performance can be assessed using either norm-referenced or criterion- 
referenced assessments. Although norm-referenced tests in reading and math have been 
used with mildly disabled students, results rarely show a direct relationship between 
treatment or intervention and changes in student achievement. Even the research with 
normal students reveals a weak relationship between classroom mstruction and student 
performance This is due to the interference of a variety of other factors such as 
mtelUgence, family Hfe, social mfluences and school organization. As weU, changes m 
achievement evidence themselves only in the long term. For these reasons it is hkely 
more useful to evaluate the "impact" of a program Hke Blueprint m the short term usmg 
criterion-referenced assessment rather than "improvement" as measured by standardized 
tests Program impact refers to short term behavioural or experiential changes m a 
student's school experience. One fairiy accessible form of criterion-referenced assessment 
that is particiilarly weU suited for evaluating the performance of students with special 
needs is goal attainment scaling. 
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This technique was developed in the clinical field to account for wide variation in 
individual client needs and in the type of treatment delivered. By utilizing a common 
scale and format, but allowing differences in content, goal attainment scaling allows for 
comparison across different presenting problems, different classroom environments, as well 
as different grades. It is -also beneficial in that it involves teacher and student in a 
shared contractual relationship. 

One champion of the use of goal attainment scaling in the education field advocates 
that staff receive • adequate training before implementing the technique (Maher 1983, 
Maher and Barbrack 1984). With Blueprint, the obvious way to do this would be through 
the school support teams with ongoing support from the resoinrce/consiilting teachers. 

(v) Parent Outcomes. 

Blueprint has a serious commitment to parental involvement in, and satisfaction 
with, the program. Parental involvement has been shown to have beneficial effects on 
student outcomes. Hayek (1987) listed the following benefits for the family: 

♦ A better understanding of the instructional needs of the child. 

♦ Active participation in the formulation of educational alternatives. 

♦ The carrying out of essential follow-up activities in the home. 

♦ The development of an understanding of the reasons for 
possible referral to special education if such a recommendation 
is made. 

While the importance of parental involvement is generally recognized at the 
iBtellectual level, the presence of some negative underlying assumptions and beliefs on 
the part of practitioners may prove to be barriers to the process of involving parents in 
the consultation and intervention process. In a recent survey of special educators, 
Donnallan and Mirenda (1984) found that "78 percent believed parents were not 
competent to choose appropriate goals for the children's education, 40 percent thought 
parents imcooperative and 38 percent thought them imrealistic" (cited in lipsky, 1989). 
It might be worthwhile to survey the perceptions of the staff most directly iavolved with 
the families about their perceptions of family involvement, not so much with an emphasis 
on their beliefs as in the study mentioned above, but rather on the problems and 
difficulties associated with involving parents, the experience of which may be the 
underlying cause of these negative beliefs. 

B. Outcomes as Changes in Organizational Capacity. 

(i) School Support Team Effectiveness. 

The team aspect of consiiltation has to do with the group problem solving process 
the members engage in with teachers. This problem-solving process can be broken down 
in two ways - into component skills or into different stages. 
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Team Skills: In a review article on Child Study Teams, Moore et al (1989) list 
three major types of skills required by teams if they are to function effectively: 1) 
interpersonal commimication skills, 2) gi'oup decision-making strategies, and 3) state-of- 
the-art evaluation techniques. 

Communication within a group whose composition is multi-disciplinary is quite 
problemati.c. One salient barrier is the presence of different technical languages 
associated with the different professions represented on such teams. For effective 
communication of information there is a need for a "common vocabulary" to evolve within 
a group (Moore et ai., 1989). As well, there is a need for the non-teaching members of 
the team to be able to communicate and negotiate with teachers in a way that facihtates 
collaboration. Research has demonstrated that teachers are more willing to adopt 
suggested procedures or interventions if they experience the following: 

♦ They are encouraged to participate in team discussions; and 
if such discussion is free of technical jargon. This is made 
possible if consultants paraphrase teachers' comments rather 
than use jargon. 

^ They are given feedback on their performance as well as social 
praise. 

4 Consultants maintain a nonjudgmental perspective. 

4 Consultants simmiarize progress and preview the agenda for 
the next discussion. 

4 Consultants demonstrate genuineness and sincere interest in 
the teacher, as well as use effective listening skills. 

Group decision-making requires a structured procedure for ensuring that the 
discussion remains focused and problem-oriented, that decisions are indeed reached, and 
that instructions about instructional intervention, information-gathering, or mutual 
assistance are clearly communicated and include accoimtabihty. Some educators suggest 
that school teams adopt the kind of group decision-making techniques prevalent in 
business and appHed social research (Moore et al., 1989). 

Knowledge about a wide range of evaluation techniques and record-keeping is 
important because of the need to assess student's academic and social skill deficits, as 
wed as the need to track student performance accurately. These are primarily techniques 
associated with single-subject methodology (e.g., curriculum-based assessment, goal 
attainment scaling). It is important for the school support teams not to neglect this 
aspect of consultation, but rather to make these tools available to teachers. 

Stages of Consxxltation: BesHes its component skills, the team problem-solving 
process can be broken down into four stages: 

4 Problem Identification - in which the team responds to a teacher's referral 
by discussing and clarifying the issues and gathering evidence/data about 
the student or classroom. 
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♦ Program Dovelopment - in which a v/orking relationship is developed with 
the teacher and an intervention plan is put together. 

♦ Program Implementation - in which the teacher, with support from the 
resource/consulting teacher, actually puts the intervention plan into practice, 

4 Program Evaluation - in which feedback is given to the teacher about how 
well (s)he implemented the interventions and whether (s)he needs further 
training in a- particular sMll area, as well as about the student's progress. 

Each of these stages represents separate tasks to be accomphshed by the team 
members, and, therefore, also represents separate outcomes, 

(ii) School Culture. 

The importance of school culture has emerged from the research on effective 
schools and school improvement. Since program implementation is central to the process 
of school improvement, it follows that school culture also impacts on schools' attempts to 
implement educational innovations such as Blueprint. Indeed, one of the foremost experts 
on program implementation in education, Michael Fullan (1983), makes the distinction 
between evaluating the ' degree of implementation" the extent to which the innovation 
is actually put into practice - from those "factors affecting implementation" - the 
contextual factors which faciUtate or inhibit the process. Two aspects of school culture 
are prominent in evaluating a program Hke Blueprint - administrative support and staff 
collegiaHty. 

Administrative Support: In one extensive study, "principal support" was judged 
to be an important perceived cause of school-based teams' effectiveness (Chalfant and Van 
Dusen Pysh, 1989). A team's effectiveness depends to some degrees on the administrative 
measures taken to incorporate the team's activity into the overall activity of the school. 
Principals are pivotal in legitimizing what the school support teams are doing by 
effectively communicating what the program is about, encouraging teachers to enter the 
consultation process, and facihtating logistical matters such as arranging for release time 
so that teachers can attend meetings. 

Staff Collegiality: Of course, the onus for a positive school chmate doesn't all 
fall on the principal. In the same survey mentioned above, another important reason for 
a school team's effectiveness was the presence of "faculty support". It has been 
demonstrated that educational innovations take root not only through formal means such 
as inservice training, but also through informal means such as teachers discussing their 
work, brainstorming together and planning together. Essentially, this is a social process 
in which teachers digest the information attached to a particular program and make it 
their own by customizing and personalizing it to fit their nt eds and the way they teach. 
"Social processing" and "Collegiality" are two terms that have been used recently to label 
this process (Seashore-Louis and Dentler, 1988; Little, 1982), CoUegiahty facihtates 
program implementation in three ways: 

♦ It allows a "shared language" to develop through which teachers can talk 
to each other about a program. This shared language serves to counteract 
the technical jargon mentioned above as a problem v/ith multi-discipHnary 
teams. It also allows teachers to discuss issues relevant to the program ia 
a concrete and coherent way (Little, 1982). 

y 
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i It promotes the development of shared ways of doing things. ^ If teachers 
talk about their reasons for going to school support teams, and if this leads 
to more teachers approaching the teams, then the potential negative value 
attached to seeking help will be diminished. 

# It encourages an atmosphere of trust and collaboration. This is extremely 
important in counteracting the conflict that will inevitably occur when 
people with different roles and statuses consult. Little (1982) found in her 
research on ' effective program implementation that "teachers in more 
successful schools were more openly confident of teachers' and 
administrators' abilities to act skilfully as observers, partners and advisors". 

(iii) System Outcomes: Changes in the Referral Process. 

Proponents of integration have pointed out that when there is no classroom-based 
intervention system in place, there is the danger that students will be referred to special 
education f( r placement who are having problems but are subsequently found to be either 
ineligible for puU-out programs or who do not require the intensive programming of a 
segregated program. This system is not only expensive [in one school board it was 
estimated to cost $1200 to assess each studer.t (cited in Chalfant and Van Dusen Pysh, 
1989)], but it also removes the student from ii.^ friends and supports of the home school 
Also, children with "mild" learning and behavioural problems may end up not recewJig 
any support; these are often described as children who 'Tall through the cracks". 
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HOW TO EVALUATE 



DATA COLLECTION INSTRUMENTS 

A variety of different tj^Des of instruments are available for collecting evaluation 
data, each with its own advantages and disadvantages, as well as suitability relative to 
program research interests and resources. 

Document analysis involves extracting commonalities and differences from 
records and docinnents kept by people involved in a program. These sources of material 
can include the ofi&cial statements and handbooks put out by the board or a school, log 
books kept by teachers or support staff, and the records kept by each school-based team. 
Document analysis is very time consuming. It can be facilitated by having all the 
participants use a similar format for record keeping. While guidelines and suggested 
formats about record keeping are being commimicated to the school support teams and 
the resource teachers participating in Blueprint, they are free to adapt these or use their 
own formats. It is possible to maintain uniformity in data collection by constructing a 
report questionnaire which can be filled out by the researcher dioring an interview with 
staff dining which staff have recourse to their files and logs while answering questions. 

Observational methods are popiilar in the research field because of the richness 
and validity of the data gathered. However, to be done well, this method requires that 
a number of observers be trained intensively and spend considerable time in the field. 
This method is less appropriate for evaluating a project as large-scale as Blueprint. 
Spotty observation yields useless data since we cannot be sure how generalizable the 
results are xmless a rigorous sampling strategy is used. 

Sui'veys, in the form of questionnaires and interviews, are the most popular form 
of data collection for evaluating large-scale programs because of the ease and speed with 
which data can be gathered and analyzed. Most of the instruments recommended for use 
in evaluating Blueprint are self-report questionnaires or semi-structured interviews. 

Questionnaires which generate a single score should be avoided. A single score is 
useful for experimental purposes, allowing the researcher to perform straightforward 
statistical computations such as correlations with other variables. However, we learn 
very httle about the content of the teachers' responses. A single score allows us only to 
identify who is or is not implementing a program, and, therefore, can easily be perceived 
as an instrument for laying blame. A more productive type of questionnaire is one in 
which the individual items themselves, or groupings of items into sub-scales, allow us to 
specify areas in which teachers are experiencing success or difficulty. 



RESEARCH DESIGN 

Results can only be interpreted in comparison to something else; for example, to 
expectations about the attitudes or behaviour being assessed, or to the performance of 
other groups or the same group at a previous time. It is necessary to plan ahead to 
identify the types of comparisons that should be included so that proper procedures can 
be adhered to dining data collection. 
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The most rigorous research designs require random samplin.g or perfectly matched 
control groups. It is possible to implement these designs in experimental situations in 
which researchers have complete control over variables as well as the samphng procedxire. 
This is next to impossible in applied research settings. The 20 schools participating in 
the pLlot year are volunteers, and we do not know if they differ in any way as a group 
from the schools that did not volunteer. Schools could be matched, but it would be 
difficult to know which criteria to use for the matching relative to the different types of 
data being collected. And finally, we are not dealing with a situation in which the effects 
of different inputs or treatments can be easily explained because the "success" of the 
program depends on a whole network of interacting program components. For example, 
student outcomes are dependent on the quality of staff development interacting with the 
effectiveness of the school support team, Literacting with the consulting skill of the 
resource teacher, interactiag with the effectiveness of the classroom interventions that are 
delivered by the teacher. In such a situation we are less interested in unravelliag causes 
and effects than we are in evaluating the relative effectiveness of each program 
component and identifying the areas in which success is being attmned or ia which 
problems are arising. 

Given the nature of Blueprint, there are three types of data comparison that would 
yield useful interpretations. 

The first has to do with comparing the results with theory-based and practice- 
based expectations. So, for example, the consultation activities and skUl use of the 
resource teachers would be compared with what the literature describes in detail as 
effective consultation. The comparison will identify areas of accomplishment or of deficit. 

The second type of comparison involves comparing the implementation performance 
at different sites, using a case-study approach to evaluation. This approach is 
appropriate for Blueprint because the factors and conditions that we expect to influence 
the implementation process exist at the school level. By considering each school 
separately as well as in relation to the others, we can describe and compare similarities 
or differences in approaches taken towards implementing the program. For example, 
questions like the following could be addressed: Are there differences in school support 
team functioning, membership, record-keeping? Do the schools differ in the types of 
student problems brought to the team? Are different interventions attempted at different 
schools? 

A case study approach allows effective formative evaluation. To identify what is 
or is not working and may involve a certain amount of accountabiUty which is always 
threatening to those being evaluated. A case study approach, however, focuses on schools 
rather than individuals, thereby placing responsibility at an organizational rather than 
personal level. It is at the organizational level that subsequent change can be more 
easily managed. 

The third type of comparison involves repeated measures. This design is 
particularly appropriate for summative outcome measin-es that would be gathered every 
year of the program implementation. Repeated measxires analysis is particularly well 
suited to identffying long-term trends. So, for example, one important outcome measure 
will be yearly referral rates. By utilizing the data for the previous few years, as well as 
collecting the data for the next five years, statistical procedures could be used to identify 
the trend ia referral rates over time. In this way it could be confirmed whether the 
program was having an effect over the referral process. 
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WHAT TO DO WITH RESULTS 



FORMATIVE AND SUMMATIVE EVALUATION 

Program evaluation can be formative, snmmativo, or both. Formative and 
snmmative evaluation refer to the decision-making purposes to which the results are put. 
Formative evaluation is ongoing during program implementation, and the data is used to 
make adjustments ir the program or to re-allocate resources. Summative evaluation 
takes place once a program has l?een fully implemented, and generally constitutes the 
final report summarizing the outcome data. 

Since Student and Community Services is interested in ongoing evaluation and 
program adjustment, and because the time frame for Blueprint is five to seven years, it 
is likely that both formative and smnmative evaluations will be carried out. 



TIMING 

An important but often neglected aspect of doing evaluation is the issue of timing 
when to gather data, and when to feed back results. Formative evaluation requires 
quick turnaround between data collection and feedback of the results. Results are not 
necessarily written up in a formal report. The aim is to get the critical information back 
as quickly as possible to the program planners or decision-makers. Summative 
evaluation, on the other hand, usually takes the form of a final report summarizing the 
final outcomes of a program that has run its full course to complete implementation. 

Because of the extended time fi:*ame of Blueprint, firom five to seven years, as well 
as the large number of sites involved, it is most reasonable to use a combination of 
formative and summative evaluation each year to track progress both in terms of degree 
of implementation (process) and outcomes. A summary report could be issued at the end 
of each school year so that feedback would inform plans for the next year. 

The evaluation of a multi-faceted program such as Blueprint will reqiiire that 
teachers, administrators and support staff complete a number of questionnaires and 
interviews. The question arises about how to do this without creating unnecessary 
anxiety in the participants. As Fullan (1985) points out, the first five months of any 
program is a time of high anxiety and confusion as participants try to learn to do new 
things. Unobtrusive measm:es are Ukely more suitable dixring this first stage. Intensive 
questionnaires and interviews could be used later in the evaluation. Logs, of course, will 
be kept on an ongoing basis. Short smrveys, whose purpose is exploratory, may be done 
at any time with care taken that they do not require too much time to complete. 

Another aspect of the timing of evaluation has to do with having reasonable 
expectations concerning program effectiveness at particular stages during program 
implementation. For example, in relation to student outcomes, the distinction has been 
made between a program's "impact" on students compared to actual "improvement" on 
their part. In other words, an effective program may, in the short term, have an^ impact 
on a student's learning experience before producing a change in achievement, which may 
only become evident in the long term. Therefore, it may be more appropriate to assess 
student outcomes in the first years of Blueprint using an impact-type measure such as 
Goal Attainment Scaling, rather than standardized achievement tests. 
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Expectations about program ouicomes should always be judged relative to a given 
stage of implementation. For example, commitment to an educational innovation is often 
seen as a "precursor" of actual involvement rather than as a "consequence" (Corbett and 
Rossman, 1989). We expect that people wiU change their attitudes before they change 
their behaviour. Corbett ai3.d Rossman report that this may be wrong. Their research 
demonstrated that commitment to an innovation does not usually develop until late in the 
implementation of a program. Teachers are willing to try new approaches in their 
classrooms (i.e., change behaviour) even if they have not fully accepted the rationale 
asso^^'uced with these new approaches. Not surprisingly, Corbett and Rosbman (1989) also 
found that long-term success of a program is often based on experiences of early success. 
Behavioural change, therefore, ,is the primary indicator of successful program 
implementation in the short term, while commitment to the program should be evaluated 
in the long term. 



MAKING SURE THE RESULTS ARE RELEVANT 

It is important to make sure that the content and criteria contained in the survey 
instruments are congruent with the program's goals and represent reasonable expectations 
about outcomes. 

A particular danger is what Little (1982) calls "measurement overkill" - the 
situation in which "evaluation methods or measures assume a degree of conceptual 
coherence, sophistication and precision not reflected in the program description". This can 
often happen when importing instruments used in other progi'ams or research contexts. 
Instruments utilized in other evaluations may be too specific to use when evaluating a 
program that is more eclectic in its approach. It is important, therefore, to estabUsh 
whether an instrument was specij&cally constructed for general use across programs or for 
a particiolar type of program. In the latter case, an instrument may be modified and 
supplemented to make it more suitable for evaluating the program under consideration. 

It is also important to look at the actual content of questionnaires, despite what 
the title says. For example, numerous "Attitudes Towards Mainstreaming" questionnaires 
have been developed. Some of these concentrate on teachers' attitudes towards different 
types of disability, including many of the most severe disabilities or handicaps. Other 
questionnaires focus on the philosophy or rationale motivatiQg the drive towards 
integration. The latter type of questionnaire content would be more relevant for a 
program such as Blueprint than the former type, since the program is aimed at students 
with mild learning disabilities or handicaps. 

Evaluation mismatch with a program as planned may also occur when one type 
of outcome is emphasized to the neglect of other, equally important, outcomes. In this 
case, a mismatch occurs because the data collected do not reflect the actual scope of the 
whole program. For example, there has been much criticism in the literature on staft 
development evaluation on the over-reUance on "opinionnaires" - that is, questionnaires 
asking participants how satisfied they were with the inservice they received. While some 
valuable information about the iaservice presentations may be gained, such evaluations 
are not adequate as "rehable or powerful predictors of actual implementation" (Little, 
1982). The results from this type of questionnaire has been shown to contain a number 
of artifacts that is, predictable responses regardless of the quality of the inservice or 
the degree of actual subsequent implementation by the participants. Teachers typically 
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report that the content was already familiar to them regardless of workshop content; 
teachers usually rate their satisfaction as average to high; and they usually agree that 
what they have learned will :-{fect how they teach their studentr. from now ou (Joyce and 
Showers, 1988). 



FEEDBACK TO PARTICIPANTS 

Results of evaluation studies have to be reported in a timely fashion and in a form 
that is easily understood by thQ participants if they are to be of any use to them. 
Because Blueprint will involve both formative and summative evaluation, and because 
different audiences are involved (administrators, teachers, support staff, parents), there 
will necessarily be a variety of report formats. A reasonable expectation is to report 
formative evaluation results to the schools on an ongoing basis, along with a final 
summative evaluation report at the end of the school year, possibly in June. 

The results of questionnaires and interviews should be reported as quickly as they 
become available to the school support teams and the teachers involved in the form of 
short reports that be could be reviewed by the personnel involved at each school These 
reports should be accompanied by discussions between researcher(s) and school staff to 
identify what issues and questions to focus on, and to identify the criteria by which to 
judge the results. Maintaining a communicative Unk between the researcher(s) and the 
staff at the schools can have two important consequences. First, the researcher(s) stay 
in touch with the participants' own understanding of, and attitude towards, the program. 
Second, giving staff the opportunity to review, interpret and use the data, will encourage 
a feeling of ownership for the evaluation research that would be absent if these results 
were commimicated in a lengthy final report from an external source. 

Summative evaluation requires a final report. The purpose of a final report is to 
present a comprehensive docxmient summarizing how well a program is doing. This 
document is intended for a larger audience than the practitioners involved in the program 
itself such as administrators involved in the program, both at the board and school level, 
trustees and interested audiences outside the school board. The final report should 
summarize all of the formative evaluation reports mentioned above, as well as the 
interpretations made by the Blueprint committee. 
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PROCESS EVALUATION OP BLUEPRINT: 
POSSIBIJE APPROACHES 



As was mentioned earlier, process evaluation is a major element in evaluating the 
implementation of policy changes. It involves gathering data about what is actually 
happening at each of the schools in implementing the program. In the case of the pilot 
year of Blueprint, it is important to describe the implementation process in order to make 
any adaptations that may seem necessary. The implementation process has ah^eady 
begun. 

During the 1989-1990 school year, inservice training was given to the resource 
teachers located at the 20 pilot schools to prepare them for taking on consulting duties 
during the 1930-1991 school yecir. Also, guidance was given to the schools to assist with 
setting up school-based teams Handbooks were developed in Hour areas - School 
Support Teams, Resource Teachers, Parental Involvement and Integration of the 
Exceptional Student. These will be sent to the pilot schools early in the 1990-91 school 
year. Ongoing inservice will be organized for the 1990-91 school year, which will also 
involve the regular teachers in these schools. Inservice has also been organized to bring 
three new schools on board. 

The purpose of process evaluation is to estabUsh how all this information actually 
gets put into practice at the different school sites. Given the Ukely degree of local 
initiative and adaptation of directives and guidelines. Blueprint will not be implemented 
in exactly the same way at aU the schools. Process evaluation attempts to describe in 
relative detail the different implementation approaches, and to capture the essence of the 
variations between the implementation site^^ (schools). 

The next section describes possible m-^asures for process evaluation under each of 
the three main program components: Consul ""ation. Intervention and Staff Development. 
Each of these program components has been subdivided into two or more of the main 
practitioner categories - school support teams, resource/consulting teachers and regular 
classroom teachers. Possible areas of investigation and questions that might be answered 
have been Usted for each section. These Lists of questions are not exhaustive; they 
represent examples of potential issues for study. 

In some cases there are suggested instnnnents that are available jfrom the 
literature. 



CONSULTATION 

School Support Teams 

4 Membersh u 

How many members are there in total? How many administrators, support 
staff, regular teachers are there? 

4 Meetings. 

How frequently are they scheduled? 
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How well attended are they? 

How successful is the team in coordinating staff schedules so that these 
meetings can occur with adequate fi'equency? 

♦ Productivity. 

How productive are the discussions? 

Do all members paHicipate, equally? 

What gets discussed"? Are any topics or issues neglected? 

What decisions are reached? 

♦ Procedures. 

Are any procedures instituted by the team to regulate its meetings and 
activities? What are they? Are they documented anywhere? 

Are all members, including the regular teachers, aware of these procedures? 

♦ Record-keeping. 

Does the team keep minutes of its meetings? 

Does the team assemble a case file for each referral? What forms are used? 
Who has access to these records, and how are they used? 

Resource/Consulting Teachers 

♦ Caseload. 

How many students do resource teachers still have in their resource rooms? 

How many students in regular classroo?ns do they consult about with the 
teacher? 

♦ Record-keeping. 

Do they keep logs concerning their consultation activities? 
What is the format and content of these logs? 

♦ Practice Profile. 

The resource teacher hcLs actually engaged in what consultation activities? 
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What proportion of time is spent in each activity? 

For this information we can use a self-report questionnaire containing a 
compreliensive and detailed list of consultation activities. Respondents can 
estimate the proportion of time spent within each category. Previous research can 
help us construct our list. For example, Tindal and Taylor-Pendergast (1989) offer 
the following taxonomy of consultation activities: 1. Written Communication, 2. 
Interpersonal Commimication, 3. Non-interactive Observation, 4. Interactive 
Testing, 5. Records Review and Material Preparation, 6. ModelUng and 
Demonstration. 

Another example is a very comprehensive taxonomy of consultation skiUs 
required by consultants in meeting the needs of exceptional children 
constructed by West and Cannon (1988) based on the responses of 100 
experts in the field. Forty-seven specific skills were organized under the 
following seven categories: 1. Consultation Theory/Models, 2. Research on 
Consultation Theory, Trainijig and Practice, 3. Personal Characteristics, 
4. Interactive Communication, 5. Collaborative Problem Soh/ing, 6. Systems 
Change, 7. Equity Issues and Values/Belief Systems. 



INTERVENTION 

School Sunnort Teams 

♦ Decision-making. 

What instructional or behavioural goals are set for the students referred? 
What intervention plans are prepared? 

What types of student evaluation are used to judge intervention success? 

♦ Methodology. 

What tools does the team use in carrying out intervention? For example, what 
types of diagnostic information were gathered? What types of evaluation of 
instructional outcome were carried out? 

Resource/Consulting Teachers 

♦ Record-keeping. 

Do they keep logs concerning the interventions they suggest to teachers, as 
well as the interventions they themselves attempt in the classroom? 

What is the format and content of these logs? 

♦ Practice Profile. 

What intervention strategies do the resource / consulting teachers prefer, and 
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for what sorts of problems? 

Regfular Teachers 

♦ Practice Profile. 

What classroom interventions does the teacher put into practice as a result 
of consultation with the school support teams and the resource teachers? 

A couple of excellent self-report questionnaires are available which contain 
lists of classroom intervention strategies suitable for children with special 
needs who are in regular classrooms. Bender and Ukeje (1989) developed 
the Bender Classroom Structure Questionnaire (BCSQ) - a 40-item. likert 
scale questionnaire hsting "research-proven insti-uctional strategies in several 
areas, including tutoring strategies, cooperative instructional gi-ouping, 
precision teaching, effective teaching behaviours, and cognitive strategies 
training". Another self-report questionnaire was developed very recently by 
Johnson and Pugach (1990) which they call the Teacher Inte:rvention 
Questionnaire (TIQ). The questionnaire contains 57 strategies that were 
specifically identified as strategies that a teacher "might use before iaitiating 
a formal referral for special education services". This fact makes this 
questionnaire particularly suitable for use with Blueprint, The items are 
presented in Appendix A. 



STAFF DEVELOPMENT 

Central Office 

♦ Inservice Training. 

Are staff satisfied with the inservice they receive? 
Are there skills areas that staff perceive as requiring more inservice? 
Is more intensive training required? By whom.? 
f Guidelixies. 

Are the school support teams and resource / consulting teachers satisfied with 
the guidelines handbooks provided to them? 

What improvements do they suggest? 

School Support Team 

♦ Public Relations within the school. 

Does the team publish policy statements for the school staff explaining the 
team's function? 

Does the team meet with staff for the same purpose? 
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Networking. 

Did the team attempt to assess ongoing needs to facilitate the consultation 
and intervention process? 

What inseruice is scheduled? 

What types of resources from outside the school are used by the team? 

What forms of networking are set up with other school support teams, and 
for what purposes? 
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OUTCOME EVALUATION OF BLUEPRrNT: 
POSSIBLE APPROACHES 



The success of a policy initiative such as Blueprint involves a number of 
interrelated outcomes. Obviously, the overall aim of the program is to provide effective 
support in the regular clc. room for students with special needs. This global aim, 
however, does not itself constitute an outcome that can be operationalized so that it coxild 
be measured/- More specificity is required; therefore. Blueprint has to be broken down 
into its constituent parts. In this section possible outcomes are organized under the three 
Blueprint program components: Consultation, Intervention and Staff Development. Once 
again, possible questions for investigation have been Usted for key variables and some 
instruments or approaches have been suggested. 



CONSULTATION 

♦ Role Perceptions, 

How much role ambiguity or role conflict do the resource I consulting teachers 
feel? 

What perceptions do regular and resource / consulting teachers have of each 
others' roles? 

What perceptions do they all have towards their role vis-d-vis the school 
support team? 

To measure role conflict. Crane and Iwardcki (1986) used a short eight- 
item Likert-scale questioxmaire. Respondents were asked whether each of 
the statements reflected their job experience. Role ambiguity was measured 
by a similar six item Likert-scale questionnaire asking them to evaluate the 
statements in the same way. The items for both questionnaires are 
presented in Appendix B. 

In evaluating Blueprint these measures of role conflict and ambiguity are 
most relevant for the resource teachers. This is not to say that role 
perceptions are not a problem for regular teachers as weU, but the degi-ee 
of role conflict and ambiguity that regular teachers feel may come from 
sources imrelated to Blueprint. Blueprint will impact the role perceptions 
of special education staff, especially the resource teachers, more than any 
other staff. 

A semi-structured interview may be the best way to find out what roles the 
different participants expected to play in the consultation process, and what 
roles they actually played. The structure of the interview may conform to 
the different stages involved in consultation, and the responsibihties 
involved with each stage. For example, we may ask the school support 
team members, the resoiurce teacher and the regular teacher separately 
what they thought their role was in the intervention programming stage of 
consultation, and then compare their answers. We can also ask them how 
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their actual responsibilities correspond to the roles they feel they should 
play in the consultation process. There are two outcomes involved here. 
One has to do with each participant's perceived actual role compared to 
their expectations. We are concerned here with estabhshing whether any 
of the participants 'feel they were asked to do more or less than they 
expected. The second type of outcome has to do with perceptions of each 
other's roles. For example, do resource teachers and regular teachers 
perceive the resource teacher's role in the same way or differently? We are 
concerned here with possible-differences in role expectations which may form 
the basis of conflicts or misunderstanding. An obvious question related to 
this issue which could hQ asked during the interview is whether the 
participants are aware of any differences in perception or expectation and, 
if so, whether any measures have been taken to resolve them. 

Satisfaction and Concerns. 

How satisfied are each of the participants (members of the school support 
team, resource / consulting and regular teachers) with the consultation process? 

What problems can they identify, and what solutions do they suggest? 

The most direct way to get this information is through an open-ended 
questionnaire or interview in which respondents are asked to state in their 
own words how satisfied they are with a program, and what their concerns 
are. 

School Support Team Effectiveness. 

Do the team members have the requisite skills to make the team function 
effectively? 

How well does the team perform in each of the four stages of consultation? 

As mentioned in the introduction, three major types of team skills have been 
identified in the Hterature: 1) interpersonal communication skills, 2) group 
decision-making strategies and 3) evaluation techniques. The four stages 
of consultation include: 1) Problem Identification, 2) Program Development, 
3) Program Implementation and 4) Program Evaluation. 

In the past, open-ended questionnaires have been used to get information 
about either teachers' satisfaction with or concerns about team effectiveness 
(see the section above on Perceptions about the Consultation and 
Intervention Process) or about the team members' own opinions about their 
team's effectiveness. In just such an open-ended questionnaire, Chalfant 
and Van Dusen Pysh (1989) asked the members of TATs to identify why 
their teams were effective. The answers fell imder three general categories: 
1) Support by the Principal, 2) Support by the schooFs faculty and 3) Team 
Attributes and Performance. This last category was farther differentiated 
into: 

• Team members were well trained in the team's operating 
procedures. 
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• Team members had a range of expertise. 
« The team leader was excellent. 

• Team members were excited about the service delivery model, 
interacted well together and worked to make the team 
effective. 

• Team members had good relationships with teachers, made a 
special effort to show their respect for their colleagues, 
remained non-threatening and tried to earn faculty trust. 

• Teams were successful in generating workable intervention 
strategies. 

Unfortunately this questionnaire did not also ask what factors were perceived to 
make a team less effective. Although it would be easy enough to ask both of these 
open-ended questions, a more informative method would be to use a semi- 
stnxctured interview. The questions making up this interview would take 
advantage of the component skill and stage taxonomies described above. In this 
way more precision can be attained in identifying which aspects of the team 
consultation are working well, and which require attention. 

Changes in the Referral Process. 

Does Blueprint affect the referral process by increasing the number of 
students teachers "refer" to the school support team. 

Does Blueprint affect the referral process by decreasing the number of 
students inappropriately referred to special education? 

One of the most important system-level outcomes is the extent to which the 
referral process will be affected by Blueprint. It is also an important 
outcome that should be included in simimative evaluation reports. To 
measure this outcome requires compiling statistics on referral rates to 
special education and to the school support teams, for several years before 
and after the Blueprint implementation began. 

Tracking referrals to the support teams (possibly broken down by school 
trimester), for each of the five years that it will take to implement Blueprint 
throughout the school board should give us a concrete indication of the 
degree to which teachers are indeed taking advantage of the program. 

Consideration of the data for referral to special education for the pilot year 
should give some indication whether the program is having an early impact 
on the referral process (formative evaluation). After five years of gathering 
these data, it will be possible to perform a time-series analysis to confirm 
that the program is having an effect in the long-run (summative evaluation). 

By trackiQg both these referral rates - to the school support teams and to 
special education - for all the schools involved, it will be possible to reach 
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some solid conclusions about the overall impact of Blueprint on the referral 
process (siimmative evaluation). By comparing rates between schools, it will 
be possible to identify schools where the process may be going awry and, 
therefore, requiring assistance (formative evaluation). 



INTERVENTION 

♦ Intervention Acceptability. 

What intervention strategies were used by regular teachers as a consequence 
of consulting with the school support team and the resource / consulting 
teacher? 

How do regular teachers rate the effectiveness of these strategies? 

How do the regular teachers rate the ease of use of these strategies? 

How well does the team and resource / consulting teacher understand and 
appreciate the regular classroom situation, in the regular teachers' opinion? 

As mentioned in the introduction, the best instrument to use in this case 
is a comprehensive questionnaire developed by Johnson and Pugach (1990) 
called the Teacher Intervention Questionnaire (see Appendix A). Also, one 
open-ended question could be added to the questionnaire, asking teachers 
what factors they consider when judging the feasibility of using particular 
intervention strategies in their classroom. 

♦ Student Outcomes. 

To what degree do the students "referred" to the teams attain their 
intervention goals? 

Are some types of goals better attained than others? 

Are there differences between schools in goals set, and in degree of goal 
attainment? 

Are there differences in goal setting and goal attainment when comparing 
grade levels? Are these differences observable across schools? 

Is there a link between goal attainment and any of the process measures, or 
outcomes involving the consultation or intervention process? 

Goal attainment is the best method for measuring student outcomes with 
a program like Blueprint in which intervention goals and plans are 
individuahzed for each student. This method of assessing student 
performance, however, requires great care in its use. The procedure 
requires specificity and a certain amount of analytic ability to set goals, set 
effective intervention plans and produce challenging yet reasonable criteria 
by which to evaluate levels of goal attainment. For example, there is the 
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danger that a program will appear more effective than another by setting 
rather unchallenging goals for its chents or students. Any evaluator using 
the results of this form of assessment will have to look closely at the goals 
and criteria set by different consultants and teachers. 

Since the school support teams will have a certain amoxmt of autonomy in 
the procedural implementation of Blueprint, it may be that goal attainment 
scahng will not be used at every school involved in the pilot year. Under 
these- circumstances it would be possible to collect some data similar, but 
not as rigorous as, goal attainment scaling data. For example, in their 
study of the functioning of TATs, Chalfant and Van Dusen Pysh (1989) used 
a "Progress Report Questionnaire" which was filled out by each team as a 
group. Each team was asked to hst the problems identified for each student 
referred to the team, the goals set for each student and the intervention 
plans decided upon. This was recorded in the team members' own words. 
Finally, the "success" of the intervention plans was evaluated according to 
three criteria: 1) "The student had achieved or nearly achieved the goals 
set" in the team members estimation, 2) "The teacher and the team agreed 
that the teacher was coping with the problem satisfactorily" and 3) "Team 
support had been withdrawn for at least 6 weeks" (Chalfant and Van Dusen 
Pysh, 1989). 

♦ Parent Outcomes. 

How aware are parents of the team consultation process? 

What types of contacts are made with the parent by the team, 
resource / consulting teacher or regular teacher? 

How are parents involved in the consultation and intervention process? 

How satisfied are the parents with this process? Do they prefer this process 
to referral to special education, possibly leading to a pull-out program? 

What problems do parents perceive, or concerns do they have, about the 
process? 

Unfortimately, there are few examples of evaluative research in which a 
measure of parental involvement has been taken. In a recent article on the 
role of parents in special education, Lipsky (1989) identifies only one 
instnnnent - The Parent/Family Involvement Index developed by Cone et 
al. (1985). This questionnaire is presented to the school-based staff working 
most knowledgeable of the family's participation in the program. It contains 
63 items requiring a "yes", "no", "not applicable" or "don't know" response. 
The questionnaire is filled out separately for the mother and the father. 
The 12 categories of types of involvement are: 1) contact with the teacher, 
2) participation in the special education process, 3) transportation, 4) 
observations at school, 5) educational activities at home, 6) attending parent 
educatiocyconsultation meetings, 7) classroom volimteering, 8. parent-parent 
contact and support, 9) involvement with administration, 10) involvement 
LDL fund raising activities, 11) involvement m advocacy groups and 12) 
disseminating information. 
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This instrument is too long and not completely appropriate for use in 
Blueprint. Ihe questionnaire is directed to parents whose children are in 
special education programs while Blueprint is directed to pre-referral 
intervention. Nevertheless, most of the items appear to be relevant, so this 
instrument can be a valuable resource for constructing a questionnaire. It 
would probably be preferable to have the parents themselves fill out the 
questionnaire. The danger here, of course, is the possibihty of a low return 
rate. ' This may be somewhat counter-acted by asking parents not only to 
indicate their involvement, but also to communicate their relative 
satisfaction about the progr^, and to identify areas of concern. To further 
encourage an acceptable return rate, it will be necessary to keep the 
questionnaire relatively short and to the point. 

Changes in the Proportion of Students Demitted from School 
Support Team Intervention and from Special Education. 

Does Blueprint affect the number of students whose problems are successfully 
resolved before referral to special education? 

Does Blueprint affect the number of exceptional students who are judged not 
to require further special education support? 

Like the changes in referral rate, this indicator is an important system- 
level outcome. We sometimes find the term "exit rate" used to label this 
indicator. Exit rate refers to the proportion of cases successfully resolved. 
Exit rates, therefore, represent a summary "score" of a program's 
intervention efi'ectiveness. We would expect a gradual increase in the exit 
rates over the years as the school support teams, resource/consulting 
teachers and regul^ teachers become progressively more effective. 



STAFF DEVELOPMENT 

♦ Attitudes Towards Lntegration. 

Do the attitudes toivards integration of those participants (members of school 
support teams, resource / consulting teachers) who experienced inseruice during 
the 1989-1990 school yean change as a result of their experience of actually 
trying to implement the program during the 1990-1991 school year? 

Do the attitudes Jowards integration of the new participants (new volunteer 
schools), change'hs a result of their inservice experience? 

Two different types of instrument have been used to measure attitudes 
towards integration (mainstreaming): Likert-scale and Response Choice 
questionnaires. 

Likert-scale Questionnaires . These questionnaires ask respondents how 
strongly they agree or disagree with a series of statements about students 
with special needs and integration. 
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One questioniiaire was developed by Joan Berryman (Berryman and Neal, 
1980). It was aimed not only at educators, but also at the general pubKc. 
The questionnaire contained 18 items, a few of which were general 
statements about kitegration but the bulk of which made reference to 
particular disabilities (e.g., "Blind students who cannot read standard 
printed material should be in regular classrooms.") While attractive because 
of its short length and possibility of use with non-educators, this 
questionnaire-ds probably not appropriate for use with Blueprint because of 
its emphasis on the more extreme types of handicap or learning disability. 

A more comprehensive questionnaire that focuses more on the rationale and 
the concerns associated with integration was developed by Garvar-Pinhas 
and SchmelMn (1989). It consists of 50 items. A factor analysis yielded 
four factors, each representing a different major concern with the process 
of integration: Academic Concerns (18 items), Socio-emotional Concerns (9 
items). Administrative Concerns (8 items), and Teacher Concerns (6 items). 
The results under hne the multi-dimensionality natin-e of educators' attitudes 
towards integration. 

Although this is the most comprehensive of the questionnaires, it may be 
a little too long to be used as a part of an evaluation of Blueprint, and it 
contains items associated with administrative and teacher corcerns that may 
be too provocative. For example, one item states that "Prmcipals beheve 
that youngsters with a handicap cannot make appropriate academic progress 
when they are integrated into the regular classroom". Principals may object 
to having this statement included. We suggest that concerns about school 
leadership, or about the negative experiences associated with integration, 
would be better obtained through open-ended interviews. In this way, also, 
concerns could be tied in with the particular school situation. 

The questionnaire that appears to be the most appropriate for use with 
Blueprint was developed by Larrivee and Cook (1979). It contains 30 items. 
The scale has a solid conceptual basis - the items were constructed with 
an eye towards a nomiber of hypothesized dimensions of teacher attitude. 
A factor analysis produced five identified factors, or dimensions: 1) general 
philosophy on integration (mainstreaming), 2) classroom behaviour of special 
needs children, 3) perceived abihty to teach special needs children, 4) 
classroom management with special needs children, 5) academic and social 
growth of the special needs child. 

The content of this questionnaire provides a good coverage of the important 
issues invoh.^ed with integration, while at the same time being not too long. 
The questionnaire items are presented in Appendix C. 

Response Choice Questionnaires. Another approach to measuring "attitudes" 
towards integration is the response choice questionnaire. In this method, 
short descriptive vignettes are presented and the respondent is asked to 
indicate what they would do about the case described. The multiple choice 
items usually consist of alternative decisions about placement for the 
student. We foimd two examples of this type of questionnaire. One is the 
Classroom Integration Questionnaire developed by Kaufinan et al. (1985) 
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for a large-scale evaluation of a mainstreaming program. The other was 
recent needs assessment questionnaire sent to Scarborough's special 
education staff (1989). Examples of the vignettes, and the response choices, 
are presented in Appendix D. 

These are very interesting questionnaires in that they do not really measure 
attitudes as much as intentions, although this term is not used to describe 
the purpose of these questionnaires. Social psychology has recently 
demonstrated that this ' distinction is more than simply a Unguistic one. 
Attitudes and intentions predict behaviour differently under different 
circumstances. For example, attitudes predict behaviour better when the 
behaviour involved does not requires a lot of effort, expertise or personal 
risk, whereas intentions predict behavioxn: better in the opposite case. It 
would seem prudent, therefore, to use both types of measurement. Results 
from the attitude questionnaires (Ldkert scale) can be used to identify needs 
around explainiug the program better (i.e., selhng the approach or rationale 
underlying integration), while results from the intention-based questionnaires 
(response choice) can be used to identify needs around encouraging teachers 
to actually take action in approaching the school support teams with 
problems. 

Practitioner Skill Development, 

To what degree do the resource / consulting teachers put into practice the 
new skills covered in the inservice training and required by their new roles? 

In what skills do the resource / consulting teachers feel they need further 
training? 

What skills do they feel were under-utilized because of the situations they 
encountered in the schools? What are these inhibiting factors? 

Outcome information about practitioner skill development can come from two 
sources. One soinrce is the process data concerning the consultation 
activities in which the resource/consulting teachers report have participated. 
Although this is "process" data concemijag the consultation process, it is 
also "outcome" data concerning the staff development process. In other 
words, the impact of the inservice training can be seen in the degree to 
which the skills covered are actually put into practice. The second soiu-ce 
Oi data is the resource/consulting teachers' own self-reports of the skills in 
which they feel competent. 

School Cxiltiire, 

What is the degree of administrative support given to implementing 
Blueprint? 

How much staff collegiality I collaboration exists in the implementation of 
Blueprint? 

Is there any relationship between these two factors and any of the program's 
outcomes? 

School culture is not really a staff development outcome but rather an 
important contextual vaiiable affecting these outcomes. 
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Information about administrative support may be gathered through a short 
interview with the principals in each school. Asking a few questions about 
this issue to the team members and the regular teachers would also be 
informative. Staff coUegiaHty can be measured by including a few questions 
concerning the extent to which discussion and shared problem-solving was 
engaged in with other teachers, and the degree to which the organizational 
structinres were in place that would allow this to happen (e.g., teachers' 
committees). 



SUMMARY 

This Kterature review and evaluation "tool kit" provides a summary of the research 
literature about evaluation of large-scale intervention policies like Scarborough's 
"Blueprint for the Future". It identifies issues and concerns that have been addressed in 
such evaluations and offers a wide range of concrete suggestions about evaluation 
methods that could be used for both formative and summative evaluation during the 
implementation of BlueprLnt. It is hoped that this report, will assist in the creation of 
a realistic plan for the Blueprint evaluation. 
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APPENDIX A 

Teacher Intervention Questionnaire 
(Johnson and Pugach, 1990) 

1. CoUect data fironi other teachers about the student's behaviour problem. 

2. Emphasize the good qualities of the student's behaviour. 

3. Talk with the student's parent about ways to work on the student's academic 
problem. 

4. Talk with other classroom teachers about ways to work on the student's academic 
problem. 

5. Adjust performance expectations in the student's problem area to increase the 
likeHhood that the student will succeed (e.g. reduce number of tasks, change 
grading criteria, etc.). 

6. Encovirage and support student's attempts at academic improvement, 

7. Establish specific consequences for appropriate student behaviour. 

8. Compile data within yovir classroom about the student's behaviour problem. 

9. Demonstrate difficult tasks for the student. 

10. Isolate the student from the class. 

11. Use peer tutors, volunteei. , or aides to work with student individually. 

12. Change the physical arrangement of the classroom. 

13. Ignore inappropriate behaviom: and attempt to change it using a positive approach. 

14. Establish specific consequences for inappropriate student behaviomr such as taking 
privileges away, assigning after-school detention, etc. 

15. Compile data within your classroom about the student's academic problem, 

16. Present same information at a slower pace or in a different sequence. 

17. Talk with school psychologist, special education teachers, counsellor, or other 
special education personnel about ways to work on the student's academic problem, 

IS. Use alternative textbook or materials. 

19 • Try different reinforcers for appropriate student behaviomr. 

20. Identify the student's behaviour problem in observable terms, 

21. Provide additional drill or practice. 
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22. Move the student vAth. academic problems to another spot in the classroom. 

23. Develop behaviour management objectives specific to the student's behaviour 
problems. 

24. Talk with the student's parent about ways to work on the student's behaviour 
problem. 

25. Discuss academic problem with student to get student's perspective. 

26. CoUect data firom the principal about the student's behaviour problem. 

27. Modify overall classroom rules. 

28. Clarify behavioral expectations to the student. 

29. CaU the class's attention to the student's inappropriate behaviour and enlist their 
support in trying to change it. 

30. Use supplementary instructional techniques (e.g. calculators, audio-recording of 
textbooks, computer-assisted instruction). 

31. Move the student with the behaviour problem to another spot in the classroom. 

32. CoUect data from other teachers about the student's academic problem. 

33. Send student to the principal. 

34. Observe the student in a situation outside the classroom. 

35. Identify the student's academic problem in observable terms. 

36. Go over lesson again with individual student. 

37. Request a staffing. 

38. Modify management routines for entire class. 

39. Clarify academic expectations to student. 

40. Talk with the principal about ways to work on the student's behaviour problem. 

41. Review student's cumulative folder. 

42. Develop instructional objectives specific to student's academic problem. 

43. Transfer student to another teacher's class. 

44. Give additional expUcit oral or written mstructions to the student. 

45. Increase consistency of response to appropriate and inappropriate behaviour. 
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,46. Use a different grouping technique (e.g. small group, individualized instruction, 
change group, etc). 

47. Have student monitor own progress toward specific academic goal. 

48. Have student monitor own progress toward specific behavioral goal. 

49. Talk with the principal about ways to wo^k on the student's academic problem. 

50. Examine factors related to student's health or family situation that may contribute 
to student's problem. 

51. Encourage and support student's attempts at improving behaviour. 

52. Give more frequent systematic corrective feedback to student's answers. 

53. Analyze subskills in the student's academic problem area and teach prerequisite 
skills first. 

54. Discuss behaviour problem with student to get student's perspective. 

55. Talk with school psychologist, special education teachers, counsellor, or other 
special education persomiel about ways to work on the student's behaviour 
problem. 

56. Collect data from the principal about the student's academic problem. 

57. Talk with other classroom teachers about ways to work on the student's behaviour 
problem. 
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APPENDIX B 

Role Conflict and Ambiguity Questionnaire 
(Crane and Iwanicki, 1986) 

A. Role Conflict. 

1. I have to do things that should be done differently. 

2. I have to work on unnecessary things. 

3. I receive an assignment without the proper manpower to complete it. 

4. I receive an assignment without adequate resources and materiads to execute it. 

5. I work with two or more groups who operatt quite differently. 

6. I have to buck a rule or policy in order to carry out an assignment. 

7. I receive incompatible requests from two or more people. 

8. I do things that are apt to be accepted by one person and not accepted by others. 

B. Role Ambiguity. 

1. I know exactly what is expected of me. 

2. I feel certain about how much authority I have. 

3. Clear, planned goals exist for my job. 

4. I know that I have divided my time properly. 

5. I know what my responsibilities are. 

6. Explanation is clear of what has to be done. 
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APPENDIX C 

Attitudes Towards Mainstreaming 
(Larrivee and Cook, 1979) 

1. Many of the things teachers do with regular students in a classroom are 
appropriate for special-needs students. 

2. The needs of handicapped students -can best be served through special, separate 
classes. 

3. A special-needs child's classroom behaviour generally requires more patience from 
the teacher than does the behaviour of a normal child. 

4. The challenge of being in a regular classroom will promote the academic growth 
of the special-needs child. 

5. The extra attention special-needs students require will be to the detriment of the 
other students. 

6. Mainstreaming offers mixed group interaction which will foster understanding and 
acceptance of differences. 

7. It is difficult to maintain order in a regular classroom that contains a special- 
needs child. 

8. Regular teachf;rs possess a great deal of the expertise necessary to work with 
special-needs students. 

9. The behaviour of special-needs students will set a bad example for the other 
students. 

10. Isolation in a special class has a negative effect on the social and emotional 
development of a special-needs student. 

11. The special-needs child will probably develop academic skills more rapidly in a 
special classroom than in a regular classroom. 

12. Most special-needs children do not make an adequate attempt to complete their 
assignments. 

13. Integration on special-needs children will require significant changes in regular 
classroom procedures. 

14. Most special-needs children are well behaved in the classroom. 

15. The contact regular-class students have vnth mains treamed students may be 
harmful. 

16. Regular-classroom teachers have siifficient training to teach children with special 
needs. 
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17. Special-needs students will monopolize the teacher's time. 

18. Mainstreaming the special-needs child will promote his/her social independence. 

19. It is hkely that a special-needs child will exhibit behaviour problems in a regular 
classroom setting. 

20. Diagnostic- prescriptive teaching is better done by resource-room or special education 
teachers than by regular-classroom teachers. 

21. The integration of special-peeds students can be beneficial for regular teachers, 

22. Special-needs children need to be told exactly what to do and how to do it. 

23. Mainstreaming is likely to have a negative effect on the emotional development of 
the special-needs child. 

24. Increased freedom in the classroom creates too much confusion. 

25. The special-needs child will be socially isolated by regular-classroom students. 

26. Parents of a special-needs child present no greater problem for a classroom teacher 
than those of a normal child. 

27. Integration of special-needs children will necessitate extensive retraining of regular 
teachers. 

28. Special-needs students should be given every opportunity to function in the regular 
classroom setting, where possible. 

29. Special-needs children are likely to create confusion in the regular classroom. 

30. The presence of special-needs students will promote acceptance of differences on 
the part of regular students. 
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APPENDIX D 

Examples of Items from Response Choice Questionnaires 
A- Classroom Intervention Questionnaire (ICaufiaian et al., 1985) 

1. Although Eric seems very bright doing science experiments and other activities 
involving manipulation of materials, he still does poorly in his reading and 
arithmetic assignments. 

2. Richard is overly dependent on the teacher. He seeks out excessive adult attention. 
He has no sense of self-direction. He never does anything without being pushed or 
prodded. 

3. Florence is immature and oversensitive, Ukely to burst into tears at the slightest 
provocation. She pouts or sulks if she can't do what she wants to do. 

Respondents are asked to choose one of the following placements for each cMld: 

♦ In regular classroom 

♦ In regular classroom all day with supplemental materials and 
advice 

♦ In regular classroom part of the day with supplemental 
materials and advice 

♦ In special class all day 

♦ Not for pubhc education 

B. Special Education Questionnaire (Scarborough Board of Education, 1989) 

1. Grade 5 student approximately one year behind in reading and spelliag. Math 
okay. Well motivated. Good peer relationships. 

2. Grade 6 student approximately one and a half years behind in all academics. 
Poorly motivated. Resistant to intervention by teacher. Poor peer relationships. 

3. Grade 8 student approximately two years behind in language areas. One year 
behind in mathematics. Poorly motivated. Few fiiends. Poor self image. 

Respondents are asked to choose the most appropriate support for each child from 
the following: 

♦ Classroom modiJBcations 

♦ Resource personnel assistiag within the classroom 

♦ Withdrawal for support within the school 

♦ Withdrawal to a congregated classroom setting 
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